Full-Stokes polarization multispectral images of various stereoscopic objects

Polarization multispectral imaging (PMI) has been applied widely with the ability of characterizing physicochemical properties of objects. However, traditional PMI relies on scanning each domain, which is time-consuming and occupies vast storage resources. Therefore, it is imperative to develop advanced PMI methods to facilitate real-time and cost-effective applications. In addition, PMI development is inseparable from preliminary simulations based on full-Stokes polarization multispectral images (FSPMI). Whereas, FSPMI measurements are always necessary due to the lack of relevant databases, which is extremely complex and severely limits PMI development. In this paper, we therefore publicize abundant FSPMI with 512 × 512 spatial pixels measured by an established system for 67 stereoscopic objects. In the system, a quarter-wave plate and a linear polarizer are rotated to modulate polarization information, while bandpass filters are switched to modulate spectral information. The required FSPMI are finally calculated from designed 5 polarization modulation and 18 spectral modulation. The publicly available FSPMI database may have the potential to greatly promote PMI development and application.

Nonetheless, all PMI improvements require simulation validation based on full-Stokes polarization multispectral images (FSPMI). Obviously, it can be found that FSPMI adopted for validation are varied in current studies due to the lack of relevant databases. This phenomenon not only increases the costs and difficulties of measuring FSPMI experimentally, but also prevents researchers from efficiently comparing existing imaging methods based on uniform FSPMI. On the contrary, other publicly available databases [25][26][27] are extremely conducive to the multidimensional and extensive development of related technologies.
In this paper, we therefore publicize an FSPMI database, including four Stokes parameters, 18 spectral bands, 512 × 512 spatial pixels and 67 stereoscopic objects. The FSPMI database is carefully measured by establishing an experimental system consisting mainly of a quarter-wave plate (QWP), a linear polarizer (LP), 18 bandpass filters and a complementary metal oxide semiconductor (CMOS) detector. By switching the bandpass filters, the broadband spectral intensities of the object reflected light are modulated and compressed into multiple narrowband spectral intensities to be sequentially detected by the CMOS. Meanwhile, by rotating the fast axis of QWP and the transmission axis of LP, the four Stokes parameters of the object reflected light are modulated and compressed into a new Stokes parameter representing the total light intensity to be also detected by the CMOS. The four Stokes parameters of the object reflected light are then calculated from the light intensities detected under five polarization modulation. Various stereoscopic objects are selected for database measurement because the polarization information can reflect the surface texture of objects. The FSPMI database is likely to enhance PMI technology development in terms of simulation validation and application popularity.
Methods experimental system establishment. To measure FSPMI, we first establish an experimental system as shown in Fig. 1. The light emitted from a light source is initially reflected by an object, then passed through a reflector, lens 1, an iris, lens 2, a QWP, an LP, 18 bandpass filters, lens 3 and finally detected by a CMOS. Both lens 1 and lens 3 are imaging lenses, while lens 2 is a collimating lens. The iris is a field stop to prevent overlapping of images. The entire cage system is established in a dark room and connected by several lens tubes to further block stray light. The manufacturers, items and key parameters of each optical component established in the experimental system are clearly listed in Table 1. Obviously, the operating wavelength range (OWR) of the system is restricted by the 18 bandpass filters, and the optical propagation size is limited by the QWP with the minimum diameter of 12.7 mm. Ignoring the fluctuation of only 2 nm, the central wavelength (CWL) of the 18 bandpass filters ranges from 520 nm to 690 nm at 10 nm intervals, and the full width at half maximum (FWHM) is about 10 nm. During the establishment of the system, the object platform is fixed within the optimal working distance range of 50 mm to 140 mm from the ring illuminator. The positions of the three lenses are then carefully adjusted according to their focal length to obtain the clearest object image on the CMOS. To facilitate experimental data measurement, the QWP is mounted in a motorized precision rotation stage, the LP is mounted in a cage rotation mount, and each of the 6 adjacent bandpass filters is mounted in a 6-position filter wheel. Image measurement. System adjustment. Figure 2 illustrates the specific experimental procedure for measuring FSPMI. Firstly, the light source is turned on and preheated for about 10 min. During this period, the object placed on the fixed platform is moved and rotated slowly in the horizontal plane to optimize the object image in the field of view of CMOS. Thus, the object region of interest is imaged within the specified 512 × 512 pixel region near the CMOS center. Meanwhile, based on the pixel format of Mono8 and the fixed exposure time of 10000 μs on CMOS, the intensity of the light source is adjusted carefully so that the maximum value of the images captured under 18 bandpass filters is approximately 230. The light source adjustment is to ensure that the captured images will not be overexposed throughout the experiment. In other words, the object position and the intensity of the light source are fixed by the above operations. Image measurement. Image capture. Then, the transmission axis of LP is rotated to 45° and fixed thereafter. Select a bandpass filter to perform the following polarization measurements. That is, rotate the fast axis of QWP to 0° to capture 6 images consecutively by CMOS for further averaging, and then rotate the fast axis of QWP to 22.5°, 45°, 67.5° and 90° to similarly capture the images respectively. Notably, the 18 bandpass filters are mounted in three 6-position filter wheels. By carefully replacing the filter wheels manually, the 18 bandpass filters are switched to perform the polarization measurements separately to capture a total of 540 images. Finally, the light source is turned off and 6 images are continuously captured by CMOS to reflect the dark noise of the detector. As described above, the entire process of image acquisition for an object takes about 35 min. Generally, the above steps are repeated several hours later to capture images for the next object. In total, the required images are captured for 67 plastic objects in about 19 days. The third object is 3D-printed and glued, while the remaining 66 objects are purchased refrigerator magnets. Image preprocessing. System fluctuation and dark noise elimination. The captured images are preprocessed separately for each object to minimize the inherent influence of the experimental system. The image preprocessing is divided into four steps: system fluctuation reduction, dark noise elimination, spectral response calibration, image maximum normalization. In order to reduce the influence of system fluctuation, the 6 images captured continuously under the same bandpass filter and QWP angle are averaged to obtain a total of 90 images for each object. The 6 images captured with the light source off are also averaged to obtain dark noise for each experiment. The averaged dark image is further subtracted from each of the averaged object images to eliminate the influence of dark noise. Moreover, the pixel value less than zero in the object image with dark noise eliminated is assigned as zero to satisfy the rationality of the captured image.
Spectral response acquisition. In addition, the spectral response of the system is determined by the combination of adopted optical devices. Figure 3 successively shows the normalized spectral characteristics of the light source, reflector, lens 1, lens 2, QWP, LP, 18 bandpass filters, lens 3, CMOS detector and the entire system. Based on standard measurements provided by the manufacturer, the spectral characteristics of each optical www.nature.com/scientificdata www.nature.com/scientificdata/ device are obtained by linear interpolation to 196 wavelengths ranging from 488.41 nm to 730.12 nm and then smoothing. The spectral characteristics of all optical devices are multiplied at each wavelength to obtain the spectral response of the entire system. Meanwhile, the spectral maximum is normalized for each optical device. In particular, the spectral response of each spectral band at all 196 wavelengths is integrated by the trapezoidal method. The 18 bandpass filters and the entire system are then normalized to the maximum integral value across all spectral bands, respectively. The integral value of spectral response after normalization of 18 bandpass filters and the entire system is shown in Fig. 4.
Spectral response calibration and image normalization. The spectral response calibration is then performed for the object image that eliminates the system fluctuation and dark noise. Specifically, the object image under each spectral band is divided by the integral value of the spectral response of the entire system under the corresponding spectral band. Finally, image normalization is carried out for each object based on the maximum value of all 90 images. Figure 5 comprehensively shows the images captured and preprocessed for an object at 18 spectral bands and five QWP angles. The captured images are displayed well in the grayscale range of 0 to 255, while the preprocessed images are displayed well in the grayscale range of 0 to 1. Obviously, image preprocessing can significantly enhance image brightness at short spectral bands to further compensate for the brightness difference caused by the uneven spectral response of the system. Moreover, image preprocessing can be further improved by experimentally measuring the spectral response of the system instead of the standard measurements provided by the manufacturer.

Data records
The raw experimental images of all 67 objects are packaged into "FSPMI experimental data.zip" and deposited to the figshare 28 . The folder labeled as "Object_xx" contains images captured for each object. For example, the folder named "Object_01" contains images captured for the first object. The subfolders and image files are described in Table 2. A total of 36582 images are captured for 67 objects, accounting for 571 GB.
The processed image data of all 67 objects is packaged into "FSPMI dataset.zip" and also deposited to the figshare 28 . The image data for each object is stored separately in a file labeled as "Object_xx.mat". For example, the file labeled as "Object_01.mat" stores image data for the first object. Table 3 lists the variable names, sizes and corresponding descriptions of the data in each object file. The data in each file includes a set of spectral bands, five sets of preprocessed images, four sets of Stokes images, two sets of polarization angle images and three sets of polarization degree images.   Table 3. The processed image data in each file labeled as "Object_xx.mat".

Technical Validation
The FSPMI dataset is intercepted by 400×400 spatial pixels and further applied to validate several existing compressive full-Stokes PMI methods [17][18][19] to demonstrate data reliability. These compressive full-Stokes PMI methods differ in terms of experimental systems and reconstruction methods. To modulate and compress the four Stokes parameters, the compressive full-Stokes polarization and flexible multispectral four-dimensional imaging (CFPMI) method 17 employs a QWP, an LP and an LCTF. Furthermore, the CFPMI method accurately solves S 0 and then reconstructs S 1 , S 2 , S 3 based on CS theory 14 . The four-dimensional compressed spectropolarimetric imaging (FDCSPI) method 18 adopts a QWP and an LCTF to modulate and compress the four Stokes parameters. Moreover, the FDCSPI method firstly reconstructs S 1 , S 2 , S 3 based on CS theory and then solves S 0 . The full polarization-compressed multispectral imaging (FPCMI) method 19 also modulates and compresses the four Stokes parameters by a QWP and an LCTF. Nevertheless, the FPCMI method first reconstructs S 0 and then S 1 , S 2 , S 3 based on feature scaling and CS theory.
The above methods are validated for all 67 objects in the FSPMI dataset. For each object data, three polarization modulation is performed in the CFPMI, FDCSPI and FPCMI methods. In the CFPMI method, the transmission axis of LP is set to 45°, 135° and 135° respectively, while the fast axis of QWP is randomly set to the same angle in the first two modulation and is changed in the third modulation. In the FDCSPI and FPCMI methods, three angles are randomly set for the fast axis of QWP. In a word, the fast axis angle of QWP is randomly set for both the three methods and 67 object data. Similarly, the transmission axis of linear polarization of the LCTF incidence plane is set to 0° in the above three methods. Meanwhile, all spectral bands of interest are provided for each object by freely switching the central wavelength of the LCTF. In addition, all the reconstruction based on CS theory use the discrete W transform basis 29 and the two-step iterative shrinkage/thresholding (TwIST) algorithm 30 with iteration accuracy of 0.005. Figure 8 shows the reconstruction results of the four Stokes parameters for all 67 objects in validating the CFPMI, FDCSPI and FPCMI methods. The reconstruction results are reflected by the peak signal to noise ratio (PSNR) and structural similarity (SSIM) values averaged over 18 spectral bands. For better comparison, the PSNR and SSIM values of each Stokes parameter are averaged over 67 objects. The averaged PSNR and SSIM values of each Stokes parameter for the CFPMI, FDCSPI and FPCMI methods are shown in Fig. 9. The averaged PSNR values are all greater than 20 dB, and the averaged SSIM values are all greater than 0.6. Therefore, the reliability of FSPMI dataset is effectively validated in the CFPMI, FDCSPI and FPCMI methods.
To improve the reconstruction results, the polarization modulation strategy can be further designed based on CS theory. In addition, the diversity of the 67 objects provides flexibility for studying compressive full-Stokes PMI methods in various scenarios. Meanwhile, the richness of the 67 objects contributes to the development of machine learning-based reconstruction methods in compressive full-Stokes PMI. www.nature.com/scientificdata www.nature.com/scientificdata/

Usage Notes
The FSPMI dataset can serve as source data for both compressive polarization imaging and compressive multispectral imaging. This dataset significantly settles the problem of inconsistency and time-consuming in data acquisition, including simulation data and experimental data. Simulation and experiment are usually combined to investigate the effectiveness of imaging methods in terms of encoding strategies and reconstruction algorithms. In general, the encoding method is different in simulation and experiment. The numerical calculation performs the simulation encoding, while the imaging system performs the experimental encoding.
Therefore, in polarization imaging, the captured polarization encoded images and the derived full-Stokes parameter images can be involved in different requirements. Less captured polarization encoded images can be used as experimental data to reconstruct full-Stokes parameter images. The derived full-Stokes parameter images can be used as simulation data to numerically calculate less polarization encoded images, and then to reconstruct full-Stokes parameter images. Furthermore, the reconstructed and derived full-Stokes parameter images are compared from various aspects to evaluate the imaging method. Meanwhile, the polarization angle images and polarization degree images can provide additional references for the evaluation.
Multispectral imaging mainly involves captured and derived light intensity images at 18 spectral bands. The central wavelength of 18 spectral bands ranges from 520 nm to 690 nm with 10 nm intervals and 10 nm bandwidths. Thus, the images in each spectral band are regarded as ideal narrowband images. Moreover, multispectral images can be used as simulation data to numerically calculate less spectral encoded images, and then to reconstruct multispectral images. Similarly, the reconstructed and unencoded multispectral images are compared sufficiently to evaluate the imaging method. Notably, image registration may be required to eliminate the slight drift of object images in different spectral bands caused by the position error of multiple filters.
In addition, the diversity of 67 objects ensures that the FSPMI dataset is suitable for developing machine learning-based imaging methods. The 67 objects are plastic cartoon images, including persons, animals, food, vehicles and other things. According to the object's structural complexity, researchers can test and select appropriate objects to meet the requirements of the imaging method.

Code availability
The experimental data are processed using MATLAB R2019b software to generate the FSPMI dataset. The software code and supporting files that generate the dataset are packaged into "FSPMI generation code.zip" and deposited to the figshare 28 . The file labeled as "FSPMI_generation_main.m" is the main program to process the experimental data to further generate the dataset. The file labeled as "text2trans.m" is the function required to run the main program, implementing the interpolation of the various spectral responses to the same wavelengths. Spectral response files (.txt) for all devices named by their items are provided in the folder "Optical devices", which is also called when running the main program. In addition, the toolboxes named "curvefit", "eml" and "matlab" are necessary to run the main program.